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DETAILED ACTION 

Information Disclosure Statement 

The listing of references in the specification is not a proper information disclosure 
statement. 37 CFR 1 .98(b) requires a list of all patents, publications, or other 
information submitted for consideration by the Office, and MPEP § 609 A(1) states, "the 
list may not be incorporated into the specification but must be submitted in a separate 
paper." Therefore, unless the references have been cited by the examiner on form 
PTO-892, they have not been considered. 

Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

2. Claim 1,6,11,20, and 35-36 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Papadimitriou et al. EpipolarLine Estimation and Rectification for Stereo 
Image Pairs (from IEEE transactions on Image Processing, April 1996, hereinafter 
"Papadimitriou"). 

Papadimitriou discloses a rectification process for a stereo pair of images (page 
675, Stage 4-Rectification of Stereo Pair) comprising first and second images capture 
using first and second image capture devices (figure 1, page 672, first sentence of the 
introduction). Papadimitriou further discloses determining first and second rectification 
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transformations for rectifying a respective one of the first and second images so as to 
reduce vertical disparity (page 675, Stage 4-Rectification of Stereo Pair). As stated in 

the first sentence of stage 4, the model represented by the {aj, i=0, ,5} guides a 

transformation process in which the two images of the stereo pair are resampled so that 

they become vertically aligned. The {a u i=0, 5} represent camera parameters that 

are functions of the focal lengths, the coordinates of the image centers, and the pan 
angles (page 673, last paragraph of col. 1 - equation (6) of col. 2). Statistics are defined 
in Webster's dictionary as a collection of quantitative data, thus {aj, i=0, — ,5} are read 
as statistics of the parameters (or a collection of quantitative data of the parameters) of 
the stereoscopic image capturing device. Therefore Papadimitriou further discloses 
using statistics of the parameters of the stereoscopic image capture device in the 
determination of the rectification transformations. 

Regarding claim 6, Papadimitriou discloses that the parameters relate to the first 
and second image capture devices (section II Epipolar Geometry beginning on page 
672). 

Regarding claim 1 1 , Papadimitriou discloses that the transformation process 
resamples the images so that they become vertically aligned, thus the statistics of the 
parameters relate to the alignment (page 675, first sentence of Stage 4-Rectification of 
Stereo Pair). 

Regarding claim 20, as seen in figure 5 Papadimitriou rectifies the stereoscopic 
pair of images. 
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Claims 35-36 are similar to the limitations discussed above, thus the same 
arguments used above apply equally to claims 35-36. 



3. Claims 1,14,22-24,33-34,and 39-40 are rejected under 35 U.S.C. 102(b) as being 
anticipated by IEEE article by Courtney et al. A Hardware Architecture for Image 
Rectification and Ground Plane Obstacle Detection (hereinafter "Courtney")(discussed 
in the Applicant's prior art on pages 8-9). 

Regarding claim 22, as stated by the Applicants on pages 8-9 of the 
specification, Courtney is one of the systems shown in the prior art system of figure 3(b) 
involving a rectification system that determines transformations to rectify the images to 
reduce vertical disparity. Courtney discloses determining rectification transforms to 
reduce vertical disparity (section 2, page 24) and further states that an image may be 
transformed so that the epipolars are aligned to rasters in each image in the other to 
simulate a parallel camera setup (page 23, first paragraph of the introduction). 

Regarding claim 23, Courtney discloses that the matrix elements of the 
transforms may be computed from camera calibration parameters including aspect 
ratios, focal lengths, etc. (last paragraph of section 2). Statistics are defined in 
Webster's dictionary as a collection of quantitative data, thus these camera calibration 
parameters can be read as statistics of the parameters of the image capture device. 
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Regarding claim 24, the camera calibration parameters are used for the 
alignment of the images thus they relate to the alignment of the first image capture 
device relative to the second image capture device. 

Claim 24 discloses all of the limitations of claim 1, thus the same arguments used 
for the rejection of claim 24 apply equally to the rejection of claim 1 . 

Claim 14 is similar to claim 22, thus the same arguments used for the rejection of 
claim 22 apply equally to claim 14. 

Regarding claim 33, as seen in figure 1 Courtney rectifies the images. 
Regarding claim 34, as seen in figure 2 Courtney discloses producing an output 
stereoscopic image. 

Claims 39-40 are similar to the limitations discussed above thus the same 
arguments used for the above claims apply equally to claims 39-40. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Papadimitriou et al. EpipolarLine Estimation and Rectification for Stereo Image Pairs 
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(from IEEE transactions on Image Processing, April 1996, hereinafter "Papadimitriou") 
in view of Zhang et al. USPN 6,608,923 (hereinafter "Zhang"). 

Papadimitriou does not explicitly disclose that each rectification transformation 
comprises a horizontal shear and scaling component, and the statistics of the 
parameters of the image capture deice are in for their determination. Zhang discloses 
that if the intrinsic parameters of a camera are known, the images are calibrated and the 
using the fundamental matrix from these parameters the images can be rectified 
through a transformation process (col. 6 lines 55-65). Furthermore as seen in figure 7 
the fundamental matrix 700 is sent to a shearing transform that scales the images in a 
horizontal direction. The process of Zhang is beneficial in that it is more efficient than 
the prior art systems and allows for rectifying images with optimally reduced distortion 
(col. 1 lines 63-67). 

Papadimitriou and Zhang are analogous art because they are from the same field 
of endeavor of rectification of stereoscopic images. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to combine Papadimitriou and Zhang to use a horizontal shear and scaling 
component determined by the statistics of the parameters of the imaging devices. The 
motivation for doing so is that it would reduce distortion in the system of Papadimitriou. 
Thus it would have been obvious to one of ordinary skill in the art to combine 
Papadimitriou and Zhang to obtain the invention as specified in claim 2. 
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6. Claims 7-10, and 37-38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Papadimitriou et al. EpipolarLine Estimation and Rectification for 
Stereo Image Pairs (from IEEE transactions on Image Processing, April 1996, 
hereinafter "Papadimitriou"). 

Claims 7-10 are dependent on claim 6 (rejected above by Papadimitriou). 
Papadimitriou does not explicitly disclose that the statistics comprise the mean and 
standard deviation of the focal points, or the mean and standard deviation of the 
principal points. However the mean and standard deviation are standard well known 
functions. As stated above Papadimitriou discloses that the parameters are functions of 
the focal lengths and image centers (page 673, paragraph above equation 6). Thus it 
would be obvious to one of ordinary skill to use the mean or standard deviation of these 
parameters as the function of the focal length with the motivation that these could be 
obtained easily through the preliminary calibration stage. 

Regarding claims 37-38, it is well known to use a data processor with a 
stereoscopic camera thus it would be obvious to use a data processor to carry out the 
invention. 

7. Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Papadimitriou et al. EpipolarLine Estimation and Rectification for Stereo Image Pairs 
(from IEEE transactions on Image Processing, April 1996, hereinafter "Papadimitriou") 
in view of the Applicant's admitted prior art. 
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Regarding claim 21, Papadimitriou does not explicitly disclose displaying the 
stereoscopic images on a stereoscopic display device, however it is obvious that it is 
doing so since the purpose of rectifying images is to see them better on a display 
device. 

As seen in figures 3a-3c of the Applicant's admitted prior art step 17 shows 
displaying the rectified image pair on a stereoscopic imaging device. 

Papadimitriou and the Applicant's admitted prior art are analogous art because 
they are from the same field of endeavor of rectification of stereoscopic images. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to combine Papadimitriou and the Applicant's admitted prior art to display the 
rectified images on a stereoscopic display device with the motivation that one could see 
the rectification results. 

8. Claims 15-17, and 19, 28-30, 32, and 41-42 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Courtney et al. A Hardware Architecture for Image 
Rectification and Ground Plane Obstacle Detection (hereinafter "Courtney") in view of 
Lipton et al. USPN 5,142,357 (hereinafter "Lipton") and further in view of the Applicant's 
admitted prior art. 

Regarding claims 15-16, Courtney does not explicitly disclose whether or not the 
image is a still image or part of a video image since it is mainly concerned with rectifying 
the image. However it is well known that a stereoscopic images can be created as still 
images or part of a video image. 
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Lipton discloses a stereoscopic camera that can be used for video or still images 
(as stated at least in the abstract). 

Lipton and Courtney are analogous art because they are from the same field of 
endeavor of stereoscopic imaging. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art that the invention as disclosed by Courtney could be used for a still stereoscopic 
image or a video stereoscopic image since both types of existing systems could be 
improved by rectification. Thus it would have been obvious to one of ordinary skill in the 
art to combine Lipton and Courtney to obtain the invention as specified in claims 15-16. 

Regarding claim 17, Courtney nor Lipton disclose rectifying subsequent frames 
of the stereoscopic video image using the first and second rectification transformations 
determined for the first frame of the stereoscopic video image. 

In the Applicant's admitted prior art (figures 3a-3c) the Applicant states that once 
suitable rectifying transformations have been determined from one captured image pair, 
a subsequent captured image pair acquired using the same camera set-up can be 
directly warped at step 15 using the rectifying transformations determined earlier (page 
8 lines 5-20), 

Courtney, Lipton, and the Applicant's admitted prior art are all analogous art 
because they all deal with stereoscopic cameras. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to combine Courtney, Lipton, and the Applicant's admitted prior to rectify 
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subsequent frames of the stereoscopic video image using the first and second 
rectifications used for the first frame. 

Regarding claim 19, it would be obvious to carry out the rectification of each 
frame of a video image using the method of Courtney since each frame would need to 
be rectified to improve the stereoscopic image. 

Claim 28 (dependent on claim 22 rejected by Courtney) is similar to claim 15 thus 
the same arguments applied for claim 15 apply equally to claim 28. The same holds true 
regarding claim 29 (similar to 16), claim 30 (similar to 17), and claim 32 (similar to claim 
19). 

Regarding claims 41-42, Lipton discloses a data processor (87 of figure 12). 

9. Claims 25-27 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Courtney et al. A Hardware Architecture for Image Rectification and Ground Plane 
Obstacle Detection (hereinafter "Courtney") in view of Zhang et al. USPN 6,608,923 
(hereinafter "Zhang"). 

As discussed above Courtney discloses the limitations of claim 22. Courtney 
does not explicitly disclose determining a first and second component of each of the first 
and second rectification transformations. 

As seen in figure 7, Zhang discloses finding a first and second component of 
each of the first and second rectification transformations, the first being the similarity 
transform 712, and the second being the shearing transform 714. The process of Zhang 
is beneficial in that it is more efficient than the prior art systems and allows for rectifying 
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images with optimally reduced distortion (col. 1 lines 63-67). Similar to Courtney who 
aligns the epipoles to simulate a parallel camera geometry (first paragraph of the 
introduction), Zhang is also interested in making the epipolars lines parallel (col. 1 lines 
25-37)(thus simulating a parallel camera geometry also). 

Courtney and Zhang are analogous art because they are from the same field of 
endeavor of rectification of stereoscopic images. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to combine Courtney and Zhang to use a first component for finding the similarity 
and thus eliminating disparity and using a second component for shearing to make a 
virtual parallel camera setup. The motivation for doing so is that it would reduce 
distortion in the system of Courtney. Thus it would have been obvious to one of ordinary 
skill in the art to combine Courtney and Zhang to obtain the invention as specified in 
claim 25. 

Regarding claims 26-27, as discussed above Courtney discloses using statistics 
of the parameters of the stereoscopic image capture device and the parameters relate 
to the alignment of the cameras. 

Allowable Subject Matter 

Claims 3-5,12-13,18, and 31 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 
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Conclusion 



The prior art made of record and not relied upon is considered pertinent to 
applicants disclosure. 

Ayache et al. Rectification of Images for Binocular and Trinocular Stereovision in 
International Conference of Pattern Recognition, pages 11-16 (1998). 

Bing Kang et al. An active multibaseline stereo system with real-time image 
acquisition Tech. Rep. CMU-CS-94-167, School of Computer Science, Carnegie Mellon 
University (1994) 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to John B Strege whose telephone number is (703) 305- 
8679. The examiner can normally be reached on Monday-Friday between the hours of 
8-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (703) 308-5246. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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